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ABSTRACT 

In  a  time  of  competing  and  constrained  resources  the  value  of  intelligence  analysis  to  decision  value  is  an 
elusive  factor  to  quantify.  This  issue  is  confounded  by  ever-increasing  initiatives  to  increase  information 
sources  and  automate  the  decision  workflow.  In  order  to  effectively  evaluate  possible  changes  to  infor¬ 
mation  analysis  and  related  decision  workflows,  it  is  critically  important  to  characterize  key  features  of 
the  decision  process.  This  paper  presents  a  theoretical  Commander’s  decision  problem:  “Should  I  attack 
and  where?”  In  the  context  of  this  problem  a  model,  based  on  a  simplified  time  dilation  optimization  me¬ 
thodology,  is  developed  to  examine  the  effects  of  intelligence  analysis  strategies  on  the  decision 
workflow  and  process.  The  model  is  evaluated  using  several  initial  assumption  cases  and  the  implica¬ 
tions  of  are  discussed. 

1  INTRODUCTION 


There  are  many  challenges  to  the  decision  process  that  intelligence  analysis  is  intended  to  support.  Many 
of  these  obstacles  boil  down  to  time/speed,  quality/clarity,  and  availability/utility.  As  a  result  not  only  do 
decisions  result  in  trade-offs  there  are  implicit  tradeoffs  within  a  decision  making  process.  Decision¬ 
making  and  the  processes  neeessary  to  support  it  may  be  viewed  from  many  different  perspectives  de¬ 
pending  on  the  scope  of  the  decision.  These  perspectives  include  strategic  level  goals  and  objectives 
down  to  the  individual  decisions  and  judgments  made  by  analysts  that  support  mission  activities  and 
commanders’  intent.  While  individual  decisions  made  by  analysts  form  the  basis  and  support  for  higher 
order  objectives  and  choices,  analysts  base  their  judgments  and  individual  decisions  on  available  informa¬ 
tion. 

The  internal  process-related  tradeoff  is  one  of  acceptable  intelligence  analysis  and  rigor  against  deci¬ 
sion  timeliness  and  outcome  value.  In  this  context,  intelligence  analysis  can  be  viewed  as  a  critical  and 
scarce  resource,  one  of  which  has  been  well  documented  (MacDonald  and  Oettinger  2002;  Pappas  and 
Simon  2002).  To  the  extent  that  intelligence  analysis  is  required  and  considered  a  valued  resource,  it  fol¬ 
lows  that  quantifying  and  optimizing  the  decision  process  can  provide  direct  and  tangible  benefits  to  deci¬ 
sion  making.  Moreover  as  organizations  seek  to  deploy  tools  that  facilitate  and  automate  decision  making 
processes,  it  will  become  increasingly  important  to  understand  the  key  features  of  the  intelligence  analysis 
characteristics  and  decision-making  environment/workflow 

To  this  end,  in  this  paper  a  theoretical  Commander’s  Problem  example  problem  is  put  forward, 
around  which  a  model  is  developed  that  allows  the  evaluation  of  various  intelligence  analysis  strategies. 
The  analytical  strategies  provide  an  initial  perspective  on  modeling  analysis-effort-value  relationships  that 
can  be  used  to  characterized  features  of  an  optimal  decision  process  within  the  context  of  the  generalized 
Commander’s  Problem.  The  paper  is  organized  as  follows.  Section  2  provides  background  on  informa¬ 
tion  and  intelligence  analysis  within  the  context  of  decision  making.  Section  3  discusses  the  relationship 
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of  rigor  to  intelligence  analysis  and  decision  value.  The  Commander’s  Problem,  the  model,  and  its  evalu¬ 
ation  are  presented  in  Section  4,  followed  by  conclusion  in  Section  5. 


2  BACKGROUND 

A  historical  review  of  warfare,  particularly  those  engagements  involving  counterinsurgency  reveals  a 
noteworthy  constant  -  none  has  been  effectively  carried  out  without  a  methodology  for  gathering  and  dis¬ 
seminating  timely  and  accurate  intelligence  data  (Clark  2006).  The  ability  to  capture  large  amounts  of  da¬ 
ta  and  the  plenitude  of  modem  intelligence  information  sources  provides  a  rich  cache  of  technical  intelli¬ 
gence  e.g.  signals  and  sensors  (SIGINT  and  MASINT),  imagery  (IMINT),  as  well  and  human  and  open 
source  intelligence  (HUMINT  and  OSINT).  The  vastness  of  this  information  presents  significant  chal¬ 
lenges  to  its  management  and  exploitation  for  decision-making  processes.  While  there  are  many  tech¬ 
niques  for  managing  information  collected  and  derived  from  these  sources,  the  exploitation  of  intelligence 
assets  for  decision-making  remains  an  ongoing  challenge  complicated  by  the  ever-increasing  amounts  of 
data  collected.  Consistent  with  Herbert  Simon’s  classic  (1969)  intelligence-design-choice  model  of  the 
human  decision  phases,  intelligence  (observation  of  reality  and  collection  of  data)  and  analysis  is  at  the 
forefront  of  this  challenge. 

2.1  Information  and  Intelligence  Analysis 

Modem  military  intelligence  analysis  is  a  complex  process  that  involves  sorting  through  huge  amounts  of 
multi-format  data  (e.g.  written  and  oral  reports,  geospatial  imagery/video,  tables  of  numeric  data,  and  vid¬ 
eo  and  audio)  from  a  wide  variety  of  sources,  combining  seemingly  unrelated  events  to  constmct  an  accu¬ 
rate  depiction  of,  and  make  predictions  about, 
temporally-dynamic  situations.  Intelligence 
analysis,  as  a  form  of  abductive  reasoning 
(Schum  1987),  is  a  special  case  of  information 
analysis.  However  unlike  common  infonnation 
analysis,  intelligence  analysis  is  fomented  by  the 
additional  problems  of  secrecy  and  adversarial 
intent.  Performed  in  the  support  of  goals  that  of¬ 
ten  extend  beyond  those  of  the  analyst  and  oc¬ 
curring  in  a  context  where  the  consequence  of 
insufficient  analysis  are  substantial,  intelligence 
analysis  is  conducted  to  influence  decisions 
where  other  agents  are  directly  operating  to  af¬ 
fect  or  manipulate  the  same  decisions.  The  es¬ 
sence  of  intelligence  analysis  is  to  transform  da¬ 
ta  into  information,  something  of  value,  which 
can  be  actionable  in  the  context  of  deeision- 
making,  potentially  at  a  variety  of  levels  (e.g. 
tactical,  operational,  and  strategic). 

2.2  Intelligence  Analysis  and  Decision-making 

At  a  functional  level,  intelligence  analysts  evaluate  hypothesis  within  a  given  domain  seeking  evidence  to 
confirm  or  reject  those  hypotheses.  In  a  document  (Clawson  2008)  titled  “What  Does  (Should)  An  Ana¬ 
lyst  Do?  A  Brief  Introduction  for  New  Analysts”  produced  by  the  Naval  Surface  Warfare  Center,  analysts 
plan,  execute  and  explain.  Planning  in  this  context  is  taken  as  determining  the  question(s)  to  be  answered, 
and  to  define  the  scope  of  the  study.  In  other  words,  define  the  problem.  Execution  is  about  the  research 


Figure  1.  Conceptual  individual  decision-making  process 
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and  evaluation  itself.  Explaining  not  only  includes  conveying  the  results,  but  also  ensuring  that  the  ex¬ 
ecution  has  sufficient  rigor  to  explain  the  outcome.  The  direct  parallel  made  by  this  operational  descrip¬ 
tion  of  what  intelligence  analysts  do  and  decision-making  processes  is  apparent.  In  decision-making  one 
observes  reality  and  defines  the  problem  (plan),  gathers  support  for  or  against  an  alternative  (execute), 
and  then  makes  a  defensible  (explained)  choice  based  on  the  execution. 

Figure  1  illustrates  an  individual’s  internal  decision  process.  Reality  is  observed,  and  this  is  collected 
as  data,  which  is  then  applied  to  learning  or  directly  to  a  reasoning  activity.  The  reasoning  activity  gets 
inputs  as  either  knowledge  or  raw  data/information  and  uses  this  to  create  or  revise  an  individual’s  belief, 
which  is  directly  applied  to  judgments  and  decisions.  The  decision  may  be  moderated  external  factors 
(not  comprehensively  illustrated  in  Figure  1)  such  as  individual  propensity  for  risk,  preferences,  and  un¬ 
certainty.  The  outcome  of  the  decision  (choice)  is  evaluated  through  the  reasoning  activity  and  mani¬ 
fested  in  further  observations  of  reality.  The  process  in  Figure  1  represents  lower  level  functions  that  oc¬ 
cur  in  the  execution  of  a  broader  intelligence  cycle. 

Intelligence  analysis  is  conducted  as  part  of  a  cycle  that  generally  follows  the  activities  necessary  for 
decision-making  (Funk  and  Sorensen  2005).  This  cycle,  which  consists  of  six  significant  steps  (planning 
and  direetion,  collection,  processing  and  exploitation,  production,  dissemination,  and  utilization)  that  oc¬ 
cur  sequentially,  translate  the  need  for  intelligence  about  a  particular  aspect  of  the  battlespace,  mission,  or 
threat  into  a  knowledge-based  product  that  is  provided  to  decision-makers.  This  cycle  incorporates  intel¬ 
ligence  development,  implicitly  consistent  with  conceptual  models  of  human  judgment  and  decision¬ 
making.  Figure  2  illustrates  the  intelligence  cycle,  intelligence  development,  Hogarth’s  (1987)  judgment 
model  and  Simon’s  (1969)  decision  model.  Figure  2  horizontally  aligns  the  components  of  the  concep¬ 
tual  models  with  the  corresponding  elements  and  activities  in  the  intelligence  development  process  and 
cycle. 
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Figure  2.  Intelligence  cyde/development  and  conceptual  models  of  human  judgment/decision-making 
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While  there  are  several  other  models  of  the  intelligence  cycle  and  decision-making,  including  TPED 
(tasking,  processing,  exploitation,  dissemination),  TPPU  (task,  post,  process,  use),  TCPED  (task,  collect, 
process,  exploit,  disseminate),  and  the  ubiquitous  OODA  (observe,  orient,  decide,  act)  loop,  these  models 
describe  activities  similar  to  those  shown  in  Figure  2.  The  intelligence  decision-analysis  process  is  intrin¬ 
sically  recursive  and  cyclic  with  feedback  loops.  If  everything  is  ideal,  the  loops  inform  each  other:  feed¬ 
back  from  tactical  loops  will  guide  decisions  at  higher  loops  and  vice  versa. 

It  is  noteworthy  that  the  broader  portion  of  Figure  2  (all  but  the  topmost  parts,  across  all  4  columns) 
deals  with  the  acquisition,  collection  and  processing  of  data.  Not  to  minimize  its  importance,  choice  is  a 
relatively  small  portion  of  the  process.  As  the  Department  of  Defense  expands  its  capability  to  collect  ev¬ 
er-increasing  amounts  of  data  from  such  systems  as  real-time/near  real-time  (RT/NRT)  full  motion  video 
(FMV)  systems  e.g.  Predator,  Constant  Hawk,  and  JSTARS,  the  larger  portion  of  the  decision-making 
value  chain  is  challenged  by  the  sheer  data  volume.  The  impediments  posed  by  the  volume  of  data  are 
compounded  by  workflows  within  the  decision  process  and  the  architectures  of  the  systems  employed  to 
facilitate  those  workflows. 

3  RIGOR  AS  A  CHALLENGE  TO  BALANCING  ANALYSIS-DECISION  VALUE 

Faced  with  significant  production  pressures  (Johnson  2005)  and  expanding  data  availability  (Patterson, 
Roth  et  al.  2001),  resulting  data  overload  deluging  the  professional  analyst,  it  is  increasingly  easy  for  ana¬ 
lysts  to  be  trapped  by  shallow,  low-rigor  analysis  (e.g.,  analysts  typically  use  basic  and  unproductive 
search  strategies  according  to  Dominguez  et.  al,  2005).  Given  similar  pressures,  it  is  also  increasingly  dif¬ 
ficult  for  decision  makers  to  recognize  when  an  analysis  is  not  of  sufficient  rigor  for  a  given  decision. 
This  uncertainty  impacts  the  analysis  process  and  it  is  also  an  elusive  metric. 

When  is  enough,  enough  in  intelligence  analysis?  The  fact  that  many  analysis  processes  are  time  li¬ 
mited  suggest  that  enough  is  enough  when  time  runs  out.  However  rigor  can  take  on  additional  dimen¬ 
sions  particularly  when  large  amounts  of  data  are  involved:  were  the  right  tools  applied?  Was  sufficient 
data  explored?  Was  the  right  methodology  used?  Conventional  perspectives  for  measuring  analytical  rigor 
focus  on  identifying  gaps  between  what  was  actually  done  versus  a  prescribed  or  “standard”  method,  with 
rigor  variously  defined  as  the  “scrupulous  adherence  to  established  standards”  (Crippen,  Daniel  et  al. 
2005)  the  “application  of  precise  and  exacting  standards”  (Military  Operations  Research  Society  2006) 
and  the  “unspoken  standard  by  which  all  research  is  measured”  (Davies  and  Dodd  2002).  These  defini¬ 
tions  of  rigor  generally  speak  to  the  process. 

Within  the  structure  of  a  “rigorous”  process  there  is  an  implicit  dependence  on  the  techniques  and 
tools  utilized  within  that  process,  implying  “appropriateness.”  From  a  practical  standpoint  some  data  col¬ 
lection  or  analysis  tools/techniques  are  viewed  as  “more”  rigorous  than  others.  If  techniques  are  tools  in 
analysts’  toolbox,  then  this  is  like  saying  that  “a  saw  is  better  than  a  hammer  because  it  is  sharper.”  Con¬ 
sider  this  issue  in  the  context  of  intelligence  analysis,  which  has  been  described  as  the  process  of  “making 
inferences  from  available  data”  and  determining  “the  best  explanation  for  uncertain,  contradictory,  and/or 
incomplete  data”  (Trent,  Patterson  et  al.  2007).  The  problem  becomes  the  use  of  the  appropriate  tool(s), 
applied  to  the  appropriate  methodology,  with  appropriate  depth  or  scope.  This  selective  balance  of  tools, 
methods,  and  judgment,  which  may  vary  from  analyst  to  analyst  in  highly  collaborative,  dynamic  envi¬ 
ronments  frames  the  problem  of  rigor  in  an  analysis  process  and  underscores  the  inherent  uncertainty 
from  both  the  analysts  and  decision-makers  viewpoint. 

Some  work  has  been  conducted  on  analysis  rigor,  notably  that  of  Zelik,  Patterson,  et  al.  The  sum¬ 
mary  of  their  efforts  showed  that  providing  process  insight  to  participants  initiated  changes  in  perceptions 
of  rigor.  It  was  also  found  that  professional  analysts  tended  to  use  similar  cues  in  making  judgments  of  ri¬ 
gor;  however,  the  ways  in  which  those  cues  were  transformed  into  composite  assessments  of  rigor  were 
more  varied.  A  surprising  finding  was  that  when  study  participant  were  guided  to  focus  on  when  a  prod¬ 
uct  was  ready  to  forward  to  a  decision  maker,  this  led  to  the  development  of  a  revised  definition  of  rigor. 
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reframing  it  as  a  multi-attribute,  emergent  measure  of  suffieieney  rather  than  a  measure  of  process  devia¬ 
tion  (Zelik,  Patterson  et  al.  2007). 

As  the  changing  technological  and  social  landscape  of  intelligence  analysis  exacerbates  the  already 
difficult  task  of  detecting  when  analysis  is  of  sufficient  rigor,  it  becomes  increasingly  uncertain  for  both 
the  analyst  and  decision  maker  to  define  a  congruent  optimal  level  of  rigor.  While  it  may  be  feasible  to 
determine  nominal  measures  of  rigor  or  define  rigor  as  a  management  objective,  a  better  understanding  of 
rigor  is  necessary  before  these  constructs  can  be  effectively  applied  to  decision-related  analysis 
workflows  and  decision  support  systems. 

Fundamentally,  the  challenge  to  achieving  equilibrium  between  analysis  effort/speed  and  decision 
value  is  striking  an  appropriate,  if  not  optimal,  balance  between  the  decision  process  and  the  outcome 
value.  Given  the  decision  process  is  comprised  of  a  workflow  that  includes  data  consumption, 
processing,  and  communication/execution,  it  maybe  possible  to  tune  the  process  such  that  the  workflows 
involved  are  designed  to  exploit  the  particular  structure  of  the  decision  opportunity. 

4  DECISION-DRIVEN  OPPORTUNITIES:  MODELING  THE  DECISION-MAKING 
PROCESS 


This  section  presents  a  model  of  a  decision  life  cycle  and  the  potential  to  optimize  information  analysis  in 
order  to  maximize  decision-making  execution.  Identifying,  describing,  and  modeling  commonly  repeated 
decision-making  processes  provide  awareness  and  focus  of  the  challenges  inherent  in  the  process.  In  par¬ 
ticular  the  focus  is  on  those  features  of  the  decision  process  that  offer  opportunities  for  optimization. 

In  order  to  evaluate  possible  changes  to  current  decision  workflows,  it  is  important  to  model  the  key 
features  of  the  decision  process.  In  controlled  environments,  the  efficacy  of  the  decision-making  process 
can  be  evaluated  over  numerous  trials  and  in  a  variety  of  scenarios.  While  the  modeling  process  necessar¬ 
ily  simplifies  some  aspects  of  the  real  world  decision  process,  key  themes  of  decision  process  challenges 
are  preserved  and  analyzed.  A  spectrum  of  approaches  from  analytical  models  through  discrete-event  si¬ 
mulation  and  process  modeling  approaches  through  to  role-playing  simulations  can  be  useful  to  under¬ 
stand  the  decision-making  environment  and  workflow. 

The  interesting  feature  of  a  decision  process  is  that  time  is  explicitly  introduced  in  the  process.  In 
general,  decisions  are  continuous  processes.  Upon  inception,  the  default  decision  is  generally  the  status 
quo,  and  the  time  allowed  before  an  absolute  deadline  allows  us  to  continuously  update  that  decision.  Of 
course,  decisions  can  be  made  before  a  deadline,  and  often  the  value  of  a  decision  is  enhanced  if  provided 
early.  Along  the  way,  the  choice  of  what  data  is  analyzed  is  a  key  driver  in  the  ability  to  make  a  timely 
and  effective  decision. 

Automation  opportunities  that  address  the  decision  process  itself  are  less  well  studied  in  the  intelli¬ 
gence  domain,  compared  to  other  domains,  e.g.  commercial  manufacturing  and  finance.  In  the  latter  do¬ 
mains,  delays  in  making  a  decision  (time  to  market)  can  significantly  influence  costs  and  profits.  The  fol¬ 
lowing  section  illustrates  a  time  dilation  model  motivated  by  a  manufacturing  scenario  (Hyden  and 
Schruben  2000)  applied  to  intelligence  analysis  and  decision  making. 

4.1  A  Commander’s  Problem:  Should  I  attack  and  where? 

Although  simplified  of  important  details,  it  is  instructive  to  consider  the  lifecycle  of  a  decision  process  se¬ 
lecting  between  a  relatively  small  number  of  alternatives.  For  the  purpose  of  illustrating  a  stylized  deci¬ 
sion  process,  we  describe  a  greatly  simplified  decision  problem.  This  scenario  allows  an  examination  of 
key  features  of  the  intelligence  decision-making  process  and  highlights  some  of  the  opportunities  and 
challenges  for  improvement. 

Suppose  a  commander  on  the  ground  has  identified  two  hostile  targets.  Given  to  resource  constraints, 
the  commander  can  only  attack  one  target.  In  order  to  choose  the  target  that  has  the  greatest  level  of  tac¬ 
tical  value,  the  commander  have  decided  to  enlist  the  help  of  intelligence  analysts.  A  full  study  of  each 
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target  would  take  two  days,  although  the  analysis  on  each  target  decomposes  naturally  into  two  single  day 
blocks  of  analysis.  Due  to  the  difficulty  of  managing  the  analysis,  the  effort  assigned  to  study  the  targets 
cannot  be  delegated  with  any  finer  detail  than  a  single  day  unit  of  analysis.  Resource  constraints  at  the 
intelligence  analysis  office  limit  the  commander  to  using  only  one  analyst  per  day.  Further,  tactical  re¬ 
quirements  will  force  the  commander  to  choose  a  target  in  two  days. 

Somewhat  surprisingly,  this  simple  scenario  offers  a  rich  array  of  analytical  content  to  model  the 
challenges  in  analyzing  decision  making.  The  scenario  has  been  winnowed  enough  to  be  described  by  the 
following  simple  structure:  The  commander’s  problem  is  to  allocate  two  days  of  analysis-time  between 
four  units  of  analysis-needs  required  for  his  alternatives.  The  ultimate  decision  of  where  to  attack  is  at 
least  partially  determined  by  the  choice  of  where/how  to  allocate  intelligence  analysis  effort.  As  shown  in 
Error!  Reference  source  not  found.,  a  choice  of  Ai  implies  that  an  analyst  will  be  assigned  to  study  the 
first  analysis  block  of  target  A;  A2  implies  an  assignment  to  the  second  analysis  block  of  Target  A  and  the 
+-1- 

It  is  assumed  that  both  targets  have  a  minimal  level  of  tactical  importance.  Based  on  previous  data  or 
perhaps  the  intuition  of  the  analyst,  it  is  also  assumed  that  any  segment  of  the  study  reveals  one  additional 
marginal  unit  of  tactical  value  with  probability  p,  where  0<p<l.  It  is  further  assumed  that  the  partitioning 
of  analysis  and  targets  insures  that  this  probability  is  independent  for  each  block  of  analysis.  Allow  Ai=l 
if  the  analysis  of  Block  1  for  target  A  reveals  an  additional  threat,  and  0  otherwise.  Putting  this  together, 
yields  P(Ai=l)=p  and  P(Ai=0)=l-p  and  each  of  Ai.  A2,  Bi,  and  B2  are  Bernoulli  random  variables  with 
parameter  p.  Hence,  a  priori,  the  probability  distribution  on  the  number  of  marginal  units  of  tactical  value 
for  each  target  is  binomial  with  parameters  p  and  n=2.  So,  stated  explicitly,  the  probability  that  Target  A 
has  k  units  of  marginal  tactical  value  is  p’‘(l-p)^^'’'^  for  k=0,l,2;  and  it  is  assumed  that  Target  B  has  the 
same  distribution. 

4.2  Modeling  the  Commander’s  Decision  Problem 

Given  the  above  description,  the  task  is  to  choose  the  target  with  the  greatest  tactical  value  given  the  in¬ 
formation  provided  from  the  commander’s  intelligence  analysis.  The  third  choice,  a  common  default  of 
many  decision  scenarios,  is  to  do  nothing.  Initially  the  correct  decision  is  to  choose  a  target  when  it  has 
the  most  tactical  value,  and  to  do  nothing  in  the  case  where  the  targets  have  the  same  value.  Later,  this  as¬ 
sumption  will  be  varied  to  examine  the  sensitivity  of  the  measures  to  aforementioned  assumptions.  To 
serve  that  decision,  it  is  necessary  to  decide  what  data  to  analyze.  To  evaluate  the  quality  of  the  decision, 
a  value  of  1  will  be  assigned  whenever  a  correct  decision  is  made. 

The  decision  process  that  has  the  highest  expected  value  will  be  presumed  to  be  the  best  decision 
process,  which  in  this  weighting,  corresponds  to  maximizing  the  probability  of  a  correct  decision.  Here 
the  term  ‘decision  process’  is  used  to  describe  the  procedure  by  which  the  decision  is  output.  Since  the 
output  of  this  decision  process  is  a  function  of  stochastic  input,  it  is  necessary  to  evaluate  the  process  us¬ 
ing  tools  from  probability.  The  method  of  evaluating  the  decision  process  is  also  critical  and  can  drive  the 
choice  for  the  optimal  strategy. 


Outcome 

Correct  Decision 

Target  A  has  greater  tactical  value 

Choose  target  A 

Target  B  has  greater  tactical  value 

Choose  target  B 

Target  A  and  target  B  have  equal  tactical  value 

Do  nothing 

Table  1:  Correct  Decisions  for  a  Commander’s  Problem 


Since  both  the  labeling  of  the  targets  and  the  units  of  analysis  on  each  target  are  arbitrary,  it  is  safe  to 
assume  without  loss  of  generality,  that  the  first  unit  of  analysis  is  always  spent  on  AI.  Initially  the  com¬ 
mander  has  two  basic  choices.  Either  the  analyst  could  be  tasked  spend  two  days  studying  target  A 
(A1,A2)  or  alternatively,  tasked  to  spend  one  day  of  analysis  on  each  target  (AI,  Bl).  Careful  study  of 
the  scenario  reveals  some  significant  details  in  the  commander’s  choice  that  can  be  used  to  enrich  the 
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model  decision  analysis.  Consider,  while  the  commander  has  two  days  to  determine  the  attack,  it  is  rea¬ 
sonable  to  allow  for  an  early  decision  by  the  commander  on  the  target  choice.  That  is,  the  commander 
may  decide  the  target  after  0,  1,  or  2  days.  This  is  a  significant  enrichment  for  many  decision  making 
scenarios,  as  it  is  reasonable  to  assume  that  a  target-analysis  is  more  valuable  the  earlier  it  gets  to  the 
commander,  recognizing  that  the  outcome  of  a  quick  decision  may  result  in  a  lower  quality  outcome. 

Another  subtle  choice  is  available  in  the  allocation  of  the  analyst.  Having  revealed  the  results  of  Al, 
the  second  day  of  analysis  can  be  chosen  as  a  function  of  the  results  obtained  at  any  time  during  day  one. 
To  the  practical  point  of  optimizing  the  decision  process,  it  is  straightforward  to  automate  this  choice.  In 
this  way,  the  allocation  of  our  analytical  resources  can  be  dynamically  adjusted  to  fit  the  needs  of  a  spe¬ 
cific  decision  process,  even  in  this  simple  scenario. 

Taking  these  enrichments  together,  this  simplified  commander’s  problem  reveals  six  natural,  fully  au¬ 
tomatic,  discrete  strategies  for  analyzing  the  targets  as  shown  in  Table  2. 


Strategy 

Comments 

None.  No  target  is  analyzed. 

No  information  is  provided  to  the  commander,  but  the  at¬ 
tack  can  be  launched  immediately. 

One  day  only.  Ai 

Only  target  A  is  examined,  and  target  A  is  only  examined 
for  1  day.  However,  the  attack  can  be  launched  after  1 
day. 

Deep.  (Al,  A?) 

Target  A  is  examined  fully  for  2  days.  Target  B  is  unexa¬ 
mined.  The  attack  is  launched  after  two  days. 

Broad.  (A|,Bi) 

Both  targets  are  examined,  but  only  for  one  day  each. 

Variable  Hybrid  1  (Greedy).  A|  then 

A2  if  Al  exposes  risk,  otherwise  Bi 

Completely  study  target  A  if  preliminary  analysis  reveals  a 
threat.  Otherwise,  study  target  B. 

Variable  Hybrid  2  (Scanning).  Ai  then 

Bi  if  A]  exposes  risk,  otherwise  A2 

Study  both  targets  if  preliminary  analysis  reveals  a  threat. 
Otherwise,  completely  study  target  A. 

Table  2:  Strategies  for  a  Commander's  Problem 


Taking  the  strategies  shown  in  Table  2,  a  plot  can  be  made  of  the  expected  value  for  each  strategy  as 
a  function  of  the  probability  that  any  day  of  analysis  discovers  a  threat,  shown  in  the  figure  as  p.  Some  in¬ 
tuitive  results  become  clear  in  the  plot  shown  in  Figure  3.  As  expected,  more  analysis  generally  increases 
the  expected  value  of  the  decision  process.  An  example  is  provider  later  illustrating  whether  this  informa¬ 
tion  actually  changes  our  decision.  In  this  case,  the  uniformly  superior  strategy  is  to  simply  allocate  the 
same  amount  of  analysis  to  each  target.  The  special  place  where  p=0.5  also  highlights  that  the  policies 
converge  when  the  possibility  of  finding  a  threat  is  merely  a  coin  flip.  The  choice  of  where  to  analyze  is 
not  important  in  that  single  instance. 


Weight  of  correctly 
choosing  target  A 

Weight  of  correctly 
choosing  target  B 

Weight  of  correctly  choos¬ 
ing  to  do  nothing 

1 

1 

1 

Case  2 

1 

1 

0 

Case  3 

.25 

.25 

1 

Table  3:  Weights  of  Correct  Decisions  in  Evaluating  Decision  Process 
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p  (probability  anaiysisfinds  a  threat) 

Figure  3.  Expected  Value  of  Decision  Process  as  a  Function  of  p  (Case  1) 

This  example  becomes  more  interesting  when  our  initial  decision  output  assumptions  are  adjusted. 
For  example,  suppose  the  evaluation  of  a  correct  decision  is  changed  so  that  zero  weight  is  given  to  ‘Do 
nothing’  as  the  correct  choice.  With  this  weighting,  it  is  necessary  to  make  a  choice  between  the  two  al¬ 
ternatives,  because  simply  choosing  to  ‘Do  nothing’  does  not  enhance  the  value  of  the  decision  process. 
Figure  4  shows  what  is  the  best  decision  process  in  that  case  and  several  interesting  changes  happen. 
First,  the  shape  of  the  response  curves  change  significantly  compared  to  the  first  case  (Figure  3).  When  p 
is  very  large  or  very  small,  the  targets  are  more  likely  to  be  tied  in  importance.  Since  the  value  of  being 
correct  in  these  instances  was  removed,  the  value  of  the  decision  process  drops  to  0  as  p  gets  close  to  1  or 
0.  This  is  probably  closer  to  the  commander’s  true  decision.  This  is  because  the  targets  really  do  have 
equal  weight;  there  isn’t  any  need  to  discern  between  the  targets  based  on  tactical  value.  The  commander 
can  choose  between  the  targets  based  on  any  other  number  of  objectives,  which  themselves  could  be  the 
subject  of  further  analysis.  For  example,  the  best  path  for  the  commander  may  be  simply  to  attack  one  of 
the  targets  without  further  analysis. 

Another  interesting  phenomenon  is  that  the  value  of  dynamic  policies  emerges  in  this  case,  where  the 
‘Do  nothing’  choice  is  correct.  The  structurally  similar  ‘greedy’  and  ‘scanning’  strategies  each  are  optim¬ 
al  over  a  range  of  values  of  p.  However,  if  examined  carefully,  it  is  apparent  that  one  strategy  is  again  su¬ 
perior  over  all  values  of  p:  the  ‘deep’  strategy.  The  optimal  strategy  in  our  first  case  (Figure  3),  the 
‘broad’  strategy,  is  now  inferior  to  all  other  strategies  that  use  2  days  of  analysis.  In  fact,  the  value  is 
even  worse  for  the  ‘broad’  strategy  because  now  the  ‘  1  day’  strategy  performs  just  as  well  and  provides 
the  answer  in  one  less  day.  By  choosing  the  ‘shared’  strategy  in  this  case  the  information  value  provided 
by  the  second  day  of  analysis  has  been  lost.  In  the  special  case  where  p=0.5,  there  is  no  increased  value 
to  spending  a  second  day  of  analysis,  no  matter  where  that  day  is  spent. 

As  a  final  case  example  (Figure  5),  the  weight  of  correct  decisions  on  choosing  a  target  is  decreased 
to  0.25  while  the  value  of  correctly  “Doing  nothing”  is  returned  to  1 .  In  this  case,  being  correct  about  de¬ 
laying  an  attack  is  valued  much  more  than  choosing  a  target,  perhaps  because  the  commander’s  resources 
are  particularly  constrained.  In  this  case,  there  is  almost  no  value  in  analyzing  the  target  at  all. 


Expected  value  of  decision  process  Expected  value  of  decision  process  Expected  value  of  decision  process 
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No  Analysis  —  Single  Day  Deep 

—  Broad  Greedy  Scanning 


No  Analysis  — Single  Day 


p  (probability analysis findsa  threat) 
—  Deep 


— Broad 


0.3 

U 

O 


p  (probability  analysis  finds  a  threat)  P  (probability  analysis  finds  a  th  reat) 

Figure  4.  Expected  Value  of  Decision  Process  (Case  2) 


NoAridiysis — SifigleDay  Deep 


p  (probability  analysis  finds  a  threat) 

Figure  5.  Expected  Value  of  Decision  Process  (Case  3) 
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5  CONCLUSION 

This  research  shows  that  the  underlying  measures  assoeiated  with  evaluating  a  decision  process  ean  drive 
optimal  strategies  for  data  collection  and  evaluation.  To  the  extent  that  intelligence  analysis  is  required 
and  a  valued  resource,  the  implications  of  affecting  speed  and  effort  in  the  analysis  task  can  provide  over¬ 
all  gains  in  decision  making.  Further,  as  organizations  seek  to  increasingly  automate  both  analysis  and 
decision  making  functions  and  workflows  the  importance  of  optimizing  analysis  in  the  context  of  ade¬ 
quate  rigor  and  output  value  will  demand  methods  the  provide  quantifiable  methods  to  tuning  the  overall 
process. 

These  points  are  illustrated  through  a  Commander’s  Problem  example  and  the  development  of  a  mod¬ 
el  where  the  effects  of  different  intelligence  analysis  strategies  on  the  theoretical  Commander’s  decision 
process  could  be  assessed.  Presupposed  in  the  Commander’s  Problem  example  is  that  the  intelligence 
analyst  has  the  ability  to  measure  the  value  of  targets  and  that  those  measures  align  with  the  specific 
needs  of  the  commander.  While  the  developed  model  begins  with  simplified  assumptions  initially,  the 
simplicity  gave  the  ability  to  focus  on  the  key  elements  of  the  decision,  and  incrementally  add  those  ele¬ 
ments  that  provided  the  greatest  value  to  the  analysis.  In  this  way,  it  was  possible  to  systematically  work 
both  ends  of  the  problem,  starting  from  a  simple  controlled  process  and  adding  the  complexities  of  a  real 
situation. 

The  complexities  of  real  world  decision  problems  are  varied,  complex  and  not  necessarily  limited  to 
the  cades  applied  within  the  developed  model.  For  example,  in  cases  where  time  is  severely  constrained, 
it  may  be  necessary  for  analysts  to  commit  their  limited  time  to  rigorously  justifying  the  decision.  FIow- 
ever,  this  research  illustrates  that  these  choices  fundamentally  affect  the  optimal  design  of  a  decision 
process.  Different  approaches  provide  new  opportunities  to  exploit  the  particular  structure  of  the  decision 
process  and  optimize  data  analysis  in  order  to  drive  decisions  as  quickly  as  possible  based  on  the  correct 
metrics.  Future  work  is  planned  to  develop  a  simulation  based  on  this  model  that  will  exercise  the  model 
under  additional  assumptions  and  more  complicated  scenarios. 
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